Methods and devices for recording changes in visual stimuli observed through browser-based interfaces

ABSTRACT

The invention provides methods and devices that address problems encountered when attempting to accurately reconstruct visual stimuli being displayed to a user as they interact with online-content, typically through a browser interface. In one embodiment, the invention provides for the browser to maintain a record of selected technical parameters and relevant data that may impact the manner in which online-content is being displayed to the user, taking into consideration the current context in which the browser is being operated. In another embodiment, the invention is a device for recording events as reported from a browser interface. The events are recorded in a selected format and syntax to form a primary index of events and related outcomes which comprise the user&#39;s interface experience. In operation, the devices detects events as detected at the browser interface. Next, the devices identifies, categorizes, and filters detected events as to their relevance to the visual stimuli being presented to the user. The devices then assigns a unique identifier, preferably in the form of an alphanumeric string, to each relevant event. Furthermore, the devices records selected events into a log (along with their categorization), a unique identifier, and the time in which the event occurred. Yet another embodiment is a devices that records changes in parameters that may impact the visual stimuli.

This application is a related to and claims priority from pendingProvisional U.S. Patent Application Ser. No. (60/190,430) entitled“METHOD AND APPARATUS FOR RECORDING CHANGES IN VISUAL STIMULI OBSERVEDTHROUGH BROWSER-BASED INTERFACES”, by Crain, et al., filed Mar. 17,2000. In addition, this application is related to U.S. patentapplication Ser. No. (09/812,394), “now U.S. Pat. No. 7,200,815 entitled“METHODS AND DEVICES FOR RECONSTRUCTING VISUAL STIMULI OBSERVED THROUGHBROWSER-BASED INTERFACES OVER TIME,” by Crain, et al., filed on Mar. 19,2001, which is incorporated by reference herein in its entirety.

BACKGROUND Technical Field

The present invention relates generally to computer systems, andbrowser-based interfaces used to display online-content to a user. Morespecifically, the invention relates to methods and devices foridentifying and recording parameters that relate to visual stimulidisplayed to a user through browser-based interfaces.

Problem Statement

As near instant access to information through the information-browsermodel becomes a mainstay of global commerce, increased attention isbeing focused on the usability and clarity of purpose of online-contentas it is displayed to a user. Being able to record the visual stimulithat is being presented to the user at any given time enables the studyof how users react to that stimuli. Accordingly, the ability toaccurately and efficiently record online-content is becoming a criticalgoal the a content design process.

One barrier to studying the visual stimuli people observe whileinteracting with on-line content though a browser is the inflexibilityof the browser interface itself. It is known that online-contentcontains a variety of textual and binary (typically encompassinggraphics) data that is interpreted by a browser and displayed to a user.The manner in which this content is displayed is not only dependent onthe type of browser being used, but also the environmental context inwhich the browser is being used. For example, a web page that appearsone way when displayed in a Microsoft based browser will appear entirelydifferent when displayed in a Netscape based browser.

A number of technologies exist that can produce a transactional recordof the actual network address of online-content viewed by a user (suchas an Internet Uniform Resource Locator or URL), and that this contentcan be revisited thereby recreating the general user interfaceexperience. However, given the complexity in how online-content canactually be displayed to a user through a browser-based interface, thosewishing to recreate the user's interface experience by reconstructingwhat visual stimuli was visible at any specific point in time for thepurposes of real-time or post-analysis have few options available.Therefore, it is desired to have a system that enables the recreation ofthe visual stimuli actually viewed by a user of a browser-interface.Unfortunately, available systems fail to provide adequate solutions.

One approach is to generalize usage parameters using standards such asbrowser version and manufacturer, operating system version andmanufacturer, and display size and aspect ratio (resolution expressed interms of the visible horizontal and vertical pixels). One deficiency inthis approach is that it does not take into consideration many of thetechnical parameters that significantly impact visual stimuli. Anotherdeficiency in this approach is that it does not provide for a method ofmaintaining reconstructable data representative of visual stimuli.

Another approach is to present the user with a “fixed” set of parametersin a controlled environment with the specific purpose of gathering usageinformation for post-exposure analysis, such as in a usabilitylaboratory. One deficiency in this approach is that it does not affordthe user a “natural” interface experience that is representative of areal-world environment. In such a real-world environment, the user wouldnormally have absolute control over the size and position of the browserwindow and horizontal and/or vertical offset of the online-contentdisplayed within that window. As with other conventional approaches,specific limiting standards are imposed on a user rather than simplyrecording technical parameters as the user interacts with theonline-content. Accordingly, there is a need for methods and devices foridentifying and recording information regarding visual stimuli as thevisual stimuli is presented to users while interacting withonline-content.

SUMMARY OF THE INVENTION

The present invention provides technical advantages to the problemsencountered when attempting to accurately reconstruct visual stimulibeing displayed to a user as they interact with online-content,typically through a browser interface. In one embodiment, the inventionprovides for the browser to maintain a record of selected technicalparameters and relevant data that may impact the manner in whichonline-content is being displayed to the user, taking into considerationthe current context in which the browser is being operated.

In one embodiment, the invention is a device for interpreting specificevents as reported from a browser interface. The events are recorded ina selected format and syntax to form a primary index of events andrelated outcomes which comprise the user's interface experience. Inoperation, the devices detects events as detected at the browserinterface. Next, the devices identifies, categorizes, and filtersdetected events as to their relevance to the visual stimuli beingpresented to the user. The devices then assigns a unique identifier,preferably in the form of an alphanumeric string, to each relevantevent. Furthermore, the devices records selected events into a log(along with their categorization), a unique identifier, and the time inwhich the event occurred.

In another embodiment, the invention is a method for identifying eventsand related parameters that impact visual stimuli being displayed to auser as the user interacts with online-content, preferably using abrowser interface. The method compares preferably categorized eventsagainst known event categories (known event categories are known tocause a specific change in the visual stimuli displayed to a user).Next, the method triggers an apparatus to record relevant parametersthat have changed as a result of the event.

Yet another embodiment is a device that records changes in parametersthat may impact the visual stimuli. The device performs readings ofseveral parameters, including the horizontal position, verticalposition, and size of each instance of a browser window currently beingdisplayed, along with the network address, horizontal offset, andvertical offset of online-content current being displayed within eachbrowser window. Next, the device records these parameters in a log filein such a manner that they are related to the original event, preferablyby using an unique identifier associated with the original event.

The methods may be embodied as manufactured devices. For example, themethods may be placed on a computer readable medium, such as a computerdiskette, CD ROM, or other memory device. In addition, the methods maybe placed in a computer memory or hard-written onto a processor toenable a general computing device to be transformed into a specificcomputing machine, or specific system. A computer system may be set upas a network capable of executing any of the methods. One such networkcould be the internet, and the network could employ an applicationservice provider. In addition, the invention may be embodied as one ormore data signals that transform a general network into a task-specificnetwork (or, task specific distributed machine).

Of course, other features and embodiments of the invention will beapparent to those of ordinary skill in the art. After reading thespecification, and the detailed description of the exemplary embodiment,these persons will recognize that similar results can be achieved in notdissimilar ways. Accordingly, the detailed description is provided as anexample of the best mode of the invention, and it should be understoodthat the invention is not limited by the detailed description.Accordingly, the invention should be read as being limited only by theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects of the invention, as well as an embodiment, are betterunderstood by reference to the following detailed description. To betterunderstand the invention, the detailed description should be read inconjunction with the drawings in which:

FIG. 1 illustrates components of a browser-based interface system;

FIG. 2 shows a process of recording multiple data streams from a browserinterface;

FIG. 2 b is a time-line illustrating some selected sampled parameters;

FIG. 3 provides a block-flow diagram of a method for recording eventsand subsequent changes in visual stimuli;

FIG. 3 b illustrates a preferred embodiment of a relational blockdiagram of a system of the invention;

FIG. 3 c illustrates an alternative embodiment of a relational blockdiagram of a system of the invention;

FIG. 3 d illustrates an alternative embodiment of a relational blockdiagram of a system of the invention;

FIG. 4 illustrates a browser perspective of a display algorithm;

FIG. 4 b is a block-flow diagram of a display algorithm;

FIG. 5 illustrates a method for determining an area of child web pagesthat is displayed within a browser window;

FIG. 5 b illustrates a content algorithm;

FIG. 6 illustrates a method, used to determine the size andtwo-dimensional location of multiple child windows;

FIG. 6 b is a flow-diagram of an offset algorithm;

FIG. 7 illustrates a hierarchical structure implemented by a method fororganizing and storing measurements of parameters and other data; and

FIG. 7 b is a flow-diagram of an organization algorithm 700.

DETAILED DESCRIPTION OF A BEST MODE

The present invention provides methods and devices to identify andrecord parameters that may relate to visual stimuli displayed to a useras they interact with online-content using a browser-based interface.

Interpretative Considerations

When reading this section (Detailed Description of a Best Mode, whichdescribes an exemplary embodiment of the best mode of the invention,hereinafter “exemplary embodiment”), one should keep in mind severalpoints. First, the following exemplary embodiment is what the inventorbelieves to be the best mode for practicing the invention at the timethis patent was filed. Thus, since one of ordinary skill in the art mayrecognize from the following exemplary embodiment that substantiallyequivalent structures or substantially equivalent acts may be used toachieve the same results in exactly the same way, or to achieve the sameresults in a not dissimilar way, the following exemplary embodimentshould not be interpreted as limiting the invention to one embodiment.

Likewise, individual aspects (sometimes called species) of the inventionare provided as examples, and, accordingly, one of ordinary skill in theart may recognize from a following exemplary structure (or a followingexemplary act) that a substantially equivalent structure orsubstantially equivalent act may be used to either achieve the sameresults in substantially the same way, or to achieve the same results ina not dissimilar way.

Accordingly, the discussion of a species (or a specific item) invokesthe genus (the class of items) to which that species belongs as well asrelated species in that genus. Likewise, the recitation of a genusinvokes the species known in the art. Furthermore, it is recognized thatas technology develops, a number of additional alternatives to achievean aspect of the invention may arise. Such advances are herebyincorporated within their respective genus, and should be recognized asbeing functionally equivalent or structurally equivalent to the aspectshown or described.

Second, the only essential aspects of the invention are identified bythe claims. Thus, aspects of the invention, including elements, acts,functions, and relationships (shown or described) should not beinterpreted as being essential unless they are explicitly described andidentified as being essential. Third, a function or an act should beinterpreted as incorporating all modes of doing that function or act,unless otherwise explicitly stated (for example, one recognizes that“tacking” may be done by nailing, stapling, gluing, hot gunning,riveting, etc., and so a use of the word tacking invokes stapling,gluing, etc., and all other modes of that word and similar words, suchas “attaching”). Fourth, unless explicitly stated otherwise, conjunctivewords (such as “or”, “and”, “including”, or “comprising”for example)should be interpreted in the inclusive, not the exclusive, sense. Fifth,the words “means” and “step” are provided to facilitate the reader'sunderstanding of the invention and do not mean “means” or “step” asdefined in §112, paragraph 6 of 35 U.S.C., unless used as “means for—functioning—” or “step for—functioning—” in the Claims section.

Computer Systems as Software Platforms

A computer system (or, system) typically includes hardware capable ofexecuting machine-readable instructions, as well as the software forexecuting acts (typically machine-readable instructions) that produce adesired result. In addition, a computer system may include hybrids ofhardware and software, as well as computer sub-systems.

Hardware generally includes processor-capable platforms, such asclient-machines (also known as personal computers or servers), andhand-held processing devices (such as smart phones, personal digitalassistants (PDAs), or personal computing devices (PCDs), for example.Furthermore, hardware typically includes any physical devices that arecapable of storing machine-readable instructions, such as memory orother data storage devices. Other forms of hardware include hardwaresub-systems, including transfer devices such as modems, modem cards,ports, and port cards, for example. The way hardware is organized withina system is known as the system's architecture (discussed below).

Software includes machine code stored in memory, such as RAM or ROM, ormachine code stored on devices (such as floppy disks, or a CD ROM, forexample). Software may include executable code, an operating system, orsource or object code, for example. In addition, software encompassesany set of instructions capable of being executed in a client machine orserver—and, in this form, is often called a program or executable code.

Programs often execute in portions of code at a time. These portions ofcode are sometimes called modules or code-segments. Often, but notalways, these code segments are identified by a particular function thatthey perform. For example, a counting module (or “counting codesegment”) may monitor the value of a variable. Furthermore, theexecution of a code segment or module is sometimes called an act.Accordingly, software may be used to perform a method which comprisesacts. In the present discussion, sometimes acts are referred to as stepsto help the reader more completely understand the exemplary embodiment.

Software also includes description code. Description code specifiesvariable values and uses these values to define attributes for adisplay, such as the placement and color of an item on a displayed page.For example, the Hypertext Transfer Protocol (HTTP) is the software usedto enable the Internet and is a description software language.

Hybrids (combinations of software and hardware) are becoming more commonas devices for providing enhanced functionality and performance tocomputer systems. A hybrid is created when traditionally softwarefunctions are directly manufactured into a silicon chip—this is possiblesince software may be assembled and compiled into ones and zeros, and,similarly, ones and zeros can be represented directly in silicon.Typically, the hybrid (manufactured hardware) functions are designed tooperate seamlessly with software. Accordingly, it should be understoodthat hybrids and other combinations of hardware and software are alsoincluded within the definition of a computer system and are thusenvisioned by the invention as possible equivalent structures andequivalent methods.

Computer sub-systems are combinations of hardware or software (orhybrids) that perform some specific task. For example, one computersub-system is a soundcard. A soundcard provides hardware connections,memory, and hardware devices for enabling sounds to be produced andrecorded by a computer system. Likewise, a soundcard may also includesoftware needed to enable a computer system to “see” the soundcard,recognize the soundcard, and drive the soundcard.

Sometimes the methods of the invention may be practiced by placing theinvention on a computer-readable medium. Computer-readable mediumsinclude passive data storage, such as a random access memory (RAM) aswell as semi-permanent data storage such as a compact disk read onlymemory (CD-ROM). In addition, the invention may be embodied in the RAMof a computer and effectively transform a standard computer into a newspecific computing machine.

Data structures are defined organizations of data and enable anembodiment of the invention. For example, a data structure may providean organization of data, or an organization of executable code(executable software). Furthermore, data signals are carried acrosstransmission mediums and store and transport various data structures,and, thus, may be used to transport the invention. It should be noted inthe following discussion that acts with like names are performed in likemanners, unless otherwise stated.

Description of the Drawings

Illustrative Architecture

Many persons are familiar with the way a computer network is set up—theway a computer network is set up is technically referred to as a networkarchitecture. FIG. 1 illustrates a specific form of a networkarchitecture, which is a browser-interface architecture 100 configuredto support the invention across a network 120. A user 115 interacts withonlinecontent by issuing requests and receiving data using abrowser-interface 200, via a host computer 130. A browser-interface 200interprets, renders, and presents information to the user 115 in theform of visual stimuli. Common browser interfaces include monitors,audio speakers, keyboards, microphones, a mouse, and other forms ofaudio/visual input or output, as well as other forms of data input.Common host computers include computing machines that operate usingnetwork software, as well as Windows, Sun, and Microsoft operatingsystem, for example.

The invention operates in conjunction with the browser-interface 200 todetect events that may cause changes in the parameters (or “technicalparameters”, in the form of data) that may impact the visual stimulibeing displayed to the user 115. The invention then records thoseparameters that are relevant to the immediate visual stimuli beingdisplayed to the user 115 in a predetermined format to a data storagedevice 140. The data storage device 140 may be local to or remote fromcomputer 110. In addition, common data storage devices include, forexample, cashed memory devices, as well as electrical and photonicstorage devices. The lightening bolt between the user 115 and thecomputer 110 represents user-based eye position feedback that providesdata regarding the position on the browser interface 200 a user's eyesare looking at, as well as information about the user's eyes, such aspupil dilation.

Recording Visual Stimuli

Accordingly FIG. 2 shows a process of recording multiple data streamsfrom a browser interface as they occur over time as a recordingalgorithm 210. The recording algorithm 210 begins with a detect visualevent act 220 in which the recording algorithm 210, which may beoperating directly in the browser software, detects an event that maychange a visual stimuli observed by a user. Next, the recordingalgorithm 210 verifies that the event detected actually impacts what theuser sees at the browser interface 200 in a verify visual parameter act230. Then, if it is verified in the verify visual parameter act 230 thatthe event does in fact change the visual display at the browserinterface, then, in a record visual parameter act 240, the event isrecorded to the data storage device 140 as described below.

FIG. 2 b illustrates parameter sampling and recording in a time-lineformat. The browser-interface 200 is configured herein to recordinformation in accordance with the timeline (in milliseconds). Time isillustrated on the horizontal axis. Time shown herein is non-discrete,meaning that the time between each of the samples illustrated in FIG. 2b are not necessarily uniform, nor are they equally spaced in time, norare they necessarily time-dependent events. The vertical axis is anon-unit based axis, and is used to illustrate parameters that may bemonitored or recorded.

In one embodiment, the timeline begins when the browser-interface 200initiates a first request and begins displaying visual stimuli to theuser. As illustrated, a variety of technical parameters and data arerecorded as streams in accordance with this timeline, as thebrowser-interface 200 detects new events (typically, as the userinteracts with online-content). Parameters and other data that arerecorded in accordance with this timeline may include events, theimmediate size and two-dimensional position of the browser window, thenetwork address of all online-content immediately displayed within thebrowser window (address), the two-dimensional offset of theonline-content as it is displayed within the browser window (offset),the two-dimensional position of the pointing device (pointer), thetextual and binary objects systemically displayed within each browserwindow (window content), a graphical image of the online content as itis immediately displayed within the browser window, a graphical image ofthe entire field of online-content available in its native size andaspect ratio (as would be displayed if the user were to scroll eitherhorizontally or vertically within the browser window), and an inventoryof all objects which comprise the online content and theirtwo-dimensional position within the browser window. Other parametersthat may be recorded include the user's eye position with respect to thebrowser interface 200 and eye dilation (eye position), size and positionof a parent window (parent—discussed later), as well as size and displayinformation regarding a child window (child—discussed later). Oneskilled in the art will understand that additional parameters relevantto the immediate visual stimuli being displayed to the user may beobtainable by the browser-interface or the device on which the browseris being operated, which may depend on the context in which thebrowser-interface is being utilized, such as browser dependent interfaceoptions, operating system settings, or other environmental variables.

Event-Based Visual Stimuli Recording

FIG. 3 provides a block-flow diagram of a method for recording eventsand subsequent changes in visual stimuli, embodied as a record-on-eventalgorithm 300 (the event algorithm 300), and will be discussed shortly.FIG. 3 can be better understood in the context of relationalblock-diagrams of systems that are enabled to record visual stimuli.Herein described are three such preferred embodiments of systems,illustrated as relational-block diagrams, that may record visualstimuli.

First Illustrative Embodiment

Accordingly, FIG. 3 b illustrates a preferred embodiment of a relationalblock diagram in which a control application 372 is configured to workwith a browser-interface 200 to record changes in the visual stimulibeing displayed to a user 115 as the user 115 interacts with anonline-content, a web page 374. The web page 374 is illustrated as asingle web page, however, because the web page 374 is “framed”, itscontent is treated in software as three separate, independent web pages(referred to as the web page's children; when the web page 374 haschildren, it is referred to as a parent web page). Accordingly, thougheach “frame page” is represented discretely as the three separated webpages 380, the online content effectively combines the three separateweb pages 381, 382, and 384 (also referred individually to as “onlinecontent”), for display purposes as the online content, the web page 374.

The control program 372 creates a custom browser object 390, which inturn, is used to launch a browser interface 200 and establish a “handle”or messaging/control conduit 391 to that interface. The resultingrelationship allows the custom browser object 390 to mirror the eventstaking place within the browser interface 200 and then report theseevents to a receiving aspect 396 of the control program 372. As the userrequests information from a host computer 130 via a computer network120, the host computer 130 responds by sending individual documents ofonline-content. As these documents are received and loaded by thebrowser-interface 200, the custom browser object 390 (in conjunctionwith the control program 372) creates any number of custom page objects395, thereby creating a 1:1 ratio of web pages 381, 382, and 384 withinthe browser interface 200 to custom page objects 395 within the custombrowser object 390.

Each custom page object 395 is used to monitor events taking placewithin its corresponding page of web pages 381, 382, and 384 and thenreport these events to the receiving aspect 396 of the controlapplication 372. The receiving aspect 396 of the control application 372monitors and records events that may cause changes in the visual stimulibeing displayed to the user as new documents of online content that areinstantiated in web pages 381, 382, and 384, and writes this data to astorage device 140, and/or an external process 397.

One skilled in the art will understand that there are many methods forcreating custom objects 390, 395, and binding custom objects 390, 395 toother programs such as web pages 381, 382, and 384 via “handles” ormessaging/control conduits 391 to a specific memory address in which thetarget program resides. Each method may be dependent on both the contextin which the browser-interface is being utilized, and browser-dependentinterface options. In a preferred embodiment of an Internet WWW browsermanufactured by Microsoft Corporation® hosted on a computer running theMicrosoft Windows® operating system, the CreateObject method [setme.objbrowser=CreateObject (“InternetExplorer.Application”)] is used tocreate a custom browser object 390 and launch an instance of the browserinterface 200 such that it returns a “handle” or messaging/controlconduit 391 thereby allowing the custom browser object 390 to listen andcontrol the newly instantiated browser interface 200.

In this embodiment, the control application 372, in conjunction with thecustom browser object 390, is used to create an empty collection ofcustom page objects 395 containing variables which are in turn set tothe “handle” or messaging/control conduit 391 for each new web page 381,382, and 384 loaded within the browser interface 200 being observed bythe custom browser object 390. Once the browser-interface 200instantiates a document in an appropriate web page 381, 382, and 384,the corresponding custom page object 395 is then used to report changesin its corresponding web page 381, 382, and 384 to an aspect of thecontrol application 372 that receives these messages 396. In thispreferred embodiment, any DHTML event handle including but not limitedto, Window.OnScroll (when a user scrolls the online-content within thebrowser window) and Window.OnResize (when a user resizes the browserwindow) can be monitored by binding the DHTML event handle to functionsof the page object 395. Upon each DHTML event, any number of propertiescan be recorded from the browser window, including but not limited to,Window.Location (the network address of the online-content beingdisplayed within the browser window), Document.Body.ClientWidth (thewidth of the browser window), Document.Body.ClientHeight (the height ofthe browser window), Document.Body.ScrollWidth (the native width of theonline-content being displayed within the browser window), andDocument.Body.ScrollHeight (the native height of the online-contentbeing displayed within the browser window).

Second Illustrative Embodiment

FIG. 3 c illustrates an alternative embodiment of a relational blockdiagram in which a control application 372 is configured to work with abrowser-interface 200 to record changes in the visual stimuli beingdisplayed to a user 115 as the user 115 interacts with online-content inweb pages 381, 382, and 384. As the user requests information from ahost computer 130 via a computer network 120, the host computer 130responds by sending individual documents 380 of online-content. As thesedocuments 380 are being received and loaded by the browser-interface374, the control program initiates a process block 398 that appends amessenger applet 350 to each document 380 thereby creating a new(modified) version of the document 352 containing the messenger applet354. As these new documents are instantiated in web pages 381, 382, and384, the messenger applet 356, 357, and 358 are used to monitor eventswhich may cause changes in the visual stimuli being displayed to theuser and communicate related data to another aspect of the controlapplication on 372 that receives these messages 396, and writes thisdata to a storage device 140, and/or an external process 397.

One skilled in the art will understand that there are many methods forappending information to a document 380 as it is loaded by abrowser-interface 374 depending on both the context in which thebrowser-interface is being utilized and browser dependent interfaceoptions. In a preferred embodiment of an Internet WWW browsermanufactured by Microsoft Corporation® hosted on a computer running theMicrosoft Windows® operating system, a JavaScript function 398 is usedto call the DHTML method “CreateElement( )” upon the DocumentCompleteevent as reported by the browser-interface. In this embodiment, theJavaScript function 398 is used to embed a messenger applet 350 in theform of an ActiveX control 354, which is appended to each document ofonline-content 352.

Once the browser-interface 374 instantiates each document 380 in anappropriate web pages 381, 384, and 382, the embedded messenger applets356, 357, and 358 is then used to report these changes in itscorresponding web pages 381, 384, and 382 to a receiving aspect 396 ofthe control application 372 that receives messages from these messengerapplets 356, 357 and 358. In this preferred embodiment, any DHTML eventhandle including but not limited to, Window.OnScroll (when a userscrolls the online-content within the browser window) andWindow.OnResize (when a user resizes the browser window) can bemonitored by binding the DHTML event handle to functions of themessenger applets 356, 357 and 358. Upon each DHTML event, any number ofproperties can be recorded from the browser window, including but notlimited to, Window.Location (the network address of the online-contentbeing displayed within the browser window), Document.Body.ClientWidth(the width of the browser window), Document.Body.ClientHeight (theheight of the browser window), Document.Body.ScrollWidth (the nativewidth of the online-content being displayed within the browser window),and Document.Body.ScrollHeight (the native height of the online-contentbeing displayed within the browser window).

Third Illustrative Embodiment

FIG. 3 d illustrates an alternative embodiment of a relational blockdiagram in which a control application 372 is configured to work with abrowser-interface 200 to record changes in the visual stimuli beingdisplayed to a user 115 as the user 115 interacts with online-content,via a web page 374. As the user requests information from a hostcomputer 130 via a network 120, the host computer 130 responds bysending individual documents 380 of online-content. These documents 380are then received and loaded by the browser-interface 374 intocorresponding web pages 381, 382, and 384. In this embodiment, oneaspect of the control application 372 interrogates the browser interface396 by polling each instance of web pages 381, 382, and 384. The browserresponds to the interrogation process with corresponding data that maybe relevant to the visual stimuli being displayed to the user which canthen written to a storage device 140, or an external process 397.

One skilled in the art will understand that there are many methods forinterrogating a browser interface 396 by polling each instance of a webpage 381, 382, and 384 depending on both the context in which thebrowser-interface is being utilized and browser dependent interfaceoptions. In the embodiment of an Internet WWW browser manufactured byMicrosoft Corporations® hosted on a computer running the MicrosoftWindows® operating system, the interrogation process is first directedat the web page 374 (also known as the parent window) to detect changesin the web page 374. Using this method, any number of properties can berecorded from the browser window, including but not limited to,Window.Location (the network address of the online-content beingdisplayed within the browser window), Document.Body.ClientWidth (thewidth of the browser window), Document.Body.ClientHeight (the height ofthe browser window), Document.Body.ScrollWidth (the native width of theonline-content being displayed within the browser window), andDocument.Body.ScrollHeight (the native height of the online-contentbeing displayed within the browser window). In this embodiment, the sameprocess is used for all subsequent web pages (child windows) 381, 382,and 384.

Illustrative Event Algorithm

The process of gathering and storing parameters may be better understoodby describing a preferred embodiment of the event algorithm 300 of FIG.3. The event algorithm 300 begins in a detect event act 310, in whichthe event algorithm 300 detects in a browser an event that may changethe visual stimuli as observed by a user viewing a browser interface.Next, the event is assigned an identification in an assign ID act 320.Preferably, the ID is an alphanumeric string, but, in any event, isunique to the event detected.

After an event is assigned an ID, the event algorithm 300 determines ifthe event is one that actually changes the visual stimuli at the browserin an affect visual stimuli query 330. If the affect visual stimuliquery determines that the event is one that does not affect the visualstimuli, then the event algorithm 300 proceeds to an end act 340 and theevent algorithm 300 terminates. If, however, the event algorithm 300detects that the event is one that does affect the visual stimuli, thenthe event algorithm 300 proceeds to a query parameters act 345. Thequery parameters act 345 provides for event algorithm 300 to detect andgather the various parameters discussed and otherwise known in the artfrom their respective sources. Then, in a store parameters act 347, theparameters that were detected and gathered in the query parameters act345 are stored.

FIG. 4 illustrates the method used to determine the location and area ofa primary browser window 450 as it appears within a larger display area455 as in the screen of a display device, as described by a displayalgorithm 400 of FIG. 4 b. First, in a measure display area act 410, thetwo-dimensional measurement of the display area 455 is calculated bydetermining both the horizontal resolution 456, and the verticalresolution 457. One skilled in the art will understand that the specificprocess for determining this two-dimensional measurement is dependent onthe device on which the browser is being operated.

Next, in a measure browser display area act 420, the two-dimensionalmeasurement of the display area occupied by the primary browser window450 is determined by measuring both its horizontal size 460 and thevertical size 462, along the two-dimensional coordinates for at leastone comer 466 of the primary browser window 450 as it is immediatelydisplayed in the display area 455. One skilled in the art willunderstand that the specific process for determining thistwo-dimensional measurement and the immediate position of at least onecomer is dependent on the browser interface being used and the device onwhich it is being operated.

FIG. 5 illustrates the method for determining the area of online-content550, 560, and 570 that is displayed within a browser window 580 and 585as it may change over time, as described by the content algorithm 500 ofFIG. 5 b. As illustrated, the example online-content 550 occupies anarea of 260,000 pixels. First, in a measure act 510, the browser window580 is positioned at the top of the online-content 560 with atwo-dimensional offset of (0,0). In this position, the browser window580 is shown to display the first 120,000 pixels of the online-content560. In this immediate position, 140,000 pixels of online-content 560are not displayed to a user. In a second measure act 520, theonline-content 570 has been moved vertically within the browser window585, giving a two-dimensional offset of (0,300). In this position, thebrowser window 585 is shown to display an area of 120,000 pixels of theonline-content 570 located between the two-dimensional coordinates of(0,300), (400,300), (400,600), and (0,600). In this position, 140,000pixels of online-content 570 are not displayed to a user appearing intwo disparate areas with the first (a first hidden area 595) locatedbetween the two-dimensional coordinates of (0,0), (400,0), (0,299), and(400,299) and the second (a second hidden area 596) located between thetwo-dimensional coordinates of (0,601), (400,601), (0,650) and(400,650).

FIG. 6 illustrates a method, described in FIG. 6 b as an offsetalgorithm 600, used to determine the size and two-dimensional locationof multiple child windows 611, 612, and 614 as they might appear withina parent browser window along with the two-dimensional offset ofonline-content 660, 661, and 662 within each child window 611, 612, and614. First, the offset algorithm 600 begins with a calculate childboundaries act 610 in which the two-dimensional boundaries for eachchild window are calculated 645, 651, 646, 649, 648, 652, 655, and 658giving the area and aspect ratio of each child window 611, 612, and 614.One skilled in the art will understand that the specific process fordetermining these two-dimensional boundaries is dependent on the browserinterface being used and the device on which it is being operated.

Next, in a calculate offset act 620, the immediate two-dimensionaloffset of the online-content 660, 661, and 662 for each child window611, 612, and 614 is calculated. The two-dimensional offset 654 and 650of the onlinecontent 660, 661, and 662 is internally represented by asingle two-dimensional measurement for each child window 611, 612, and614.

FIG. 7 illustrates a hierarchical structure implemented by a method fororganizing and storing measurements of parameters and other data,illustrated in FIG. 7 b as an organization algorithm 700. Thehierarchical structure originates from the control application 372, andorganizes parameters and data that impact the immediate visual stimulibeing displayed to the user.

In a preferred embodiment, the control algorithm 700 begins in a createevent object act 710 in which the control application 372 createsobjects depending on the type of event observed. Any event-specificparameters are then recorded in a record parameters act 720. Forexample, in the case of a change in the two-dimensional location of thepointing device, a pointing device event 766 is created containing thecurrent time and the two-dimensional position of the pointing device. Inthe case of a change in the visual stimuli within one or more windows ofthe browser interface, a visual event 767 is created containing thecurrent time, an assigned event type, and the unique id assigned to theevent by the control program. A visual event 767 may require thatadditional information be recorded.

Next, in a create screen image object act 730, in the case of thebrowser window, a browser window object 768 is created containing thetwo-dimensional location of the browser window, the size of the browserwindow expressed as its two-dimensional boundaries, the two-dimensionaloffset of the online-content immediately displayed in the browserwindow, the unique identifier for the browser window, and the uniqueidentifier of the browser window's parent window (in the event thebrowser window has no parent, this property is set to a null value).These parameters are recorded in a record intermediate parameters act740.

In the case of the online-content, a create objects act 750 creates anonline content object 769 containing the unique id of the browser windowin which the content is being displayed, the network address, textualdata, and binary objects that comprise the online-content beingimmediately displayed in the browser window, the native size of theonline-content expressed as its two-dimensional boundaries are recordedalong with a graphical screen snapshots of the online-content both inits native format and visual format as bound by the browser window.These objects are recorded in a record new object parameters act 760. Inaddition, an inventory of the objects which comprise the online contentalong with their two-dimensional location is recorded. Other objects,such as child objects, and online-content objects, can be likewisecreated and recorded.

Though the invention has been described with respect to a specificpreferred embodiment, many variations and modifications will becomeapparent to those skilled in the art upon reading the presentapplication. It is therefore the intention that the appended claims beinterpreted as broadly as possible in view of the prior art to includeall such variations and modifications.

1. A method of electronically processing browser-viewable visualstimuli, comprising: detecting an event comprising changes in visualstimuli electronically presented to, observable by, and resultant from auser interacting with a browser interface; assigning an identification(ID) to the event; evaluating at least one changed parameter of thevisual stimuli caused by the event; recording the event, including thechanged parameters, for later use in reconstruction; and reconstructingthe event based on the recorded event including the changed parameters.2. The method of claim 1 further comprising querying at least one of thechanged parameters.
 3. The method of claim 1 wherein the ID is analphanumeric string.
 4. The method of claim 1 further comprisingdetermining the location and area of a primary browser window.
 5. Themethod of claim 4 further comprising determining a horizontal size and avertical size of a primary browser window as immediately displayed inthe display area, and a two-dimensional location of at least a corner ofthe primary browser window.
 6. The method of claim 1 further comprisingdetermining a location of online-content displayed within the primarybrowser window.
 7. The method of claim 6 further comprising determiningthe location of the online-content as an offset with respect to a cornerof the primary browser window.
 8. The method of claim 1 furthercomprising determining a parameter of a child window.
 9. The method ofclaim 8 further comprising determining a size of the child window. 10.The method of claim 8 further comprising determining a two-dimensionallocation of the child window relative to a primary browser window. 11.The method of claim 8 further comprising determining a position ofonline-content displayed within the child window.
 12. The method ofclaim 11 further comprising determining an offset of the online-contentrelative to a corner of the child window.
 13. The method of claim 1further comprising creating an event object.
 14. The method of claim 13further comprising recording a parameter of the event object.
 15. Themethod of claim 1 further comprising creating a screen-image object. 16.The method of claim 15 further comprising recording a parameter of thescreen-image object.
 17. The method of claim 16 further comprisingquerying the parameter.
 18. The method of claim 1 further comprising thestep of recording at least one of the evaluated parameters.
 19. Acomputer readable medium comprising instructions for: detecting an eventcomprising changes in visual stimuli electronically presented to,observable by, and resultant from a user interacting with a browserinterface; assigning an identification (ID) to the event; evaluating atleast one changed parameter of the visual stimuli caused by the event;recording the event, including the changed parameters, for later use inreconstruction; and reconstructing the event based on the recorded eventincluding the changed parameters.
 20. The computer readable medium ofclaim 19 further comprising querying at least one of the changedparameters.
 21. The computer readable medium of claim 19 wherein the IDis an alphanumeric string.
 22. The computer readable medium of claim 19further comprising instructions for determining the location and area ofa primary browser window.
 23. The computer readable medium of claim 22further comprising instructions for determining a horizontal size and avertical size of a primary browser window as immediately displayed inthe display area, and a two-dimensional location of at least a corner ofthe primary browser window.
 24. The computer readable medium of claim 19further comprising instructions for determining a location ofonline-content displayed within the primary browser window.
 25. Thecomputer readable medium of claim 24 further comprising instructions fordetermining the location of the online-content as an offset with respectto a corner of the primary browser window.
 26. The computer readablemedium of claim 19 further comprising instructions for determining aparameter of a child window.
 27. The computer readable medium of claim26 further comprising instructions for determining a size of the childwindow.
 28. The computer readable medium of claim 26 further comprisinginstructions for determining a two-dimensional location of the childwindow relative to a primary browser window.
 29. The computer readablemedium of claim 26 further comprising instructions for determining aposition of online-content displayed within the child window.
 30. Thecomputer readable medium of claim 29 further comprising instructions fordetermining an offset of the online-content relative to a corner of thechild window.
 31. The computer readable medium of claim 29 furthercomprising instructions for recording at least one of the evaluatedparameters.
 32. The computer readable medium of claim 19 furthercomprising instructions for creating an event object.
 33. The computerreadable medium of claim 32 further comprising instructions forrecording a parameter of the event object.
 34. The computer readablemedium of claim 19 further comprising instructions for creating ascreen-image object.
 35. The computer readable medium of claim 34further comprising instructions for recording a parameter of thescreen-image object.
 36. The computer readable medium of claim 35further comprising instructions for querying the parameter.